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Abstract. In this article, we study the problem of finding the longest common 
separable pattern between several permutations. We give a polynomial-time 
algorithm when the number of input permutations is fixed and show that the 
problem is NP-hard for an arbitrary number of input permutations even if 
these permutations are separable. 

On the other hand, we show that the NP-hard problem of finding the longest 
common pattern between two permutations cannot be approximated better 
than within a ratio of y/Opt (whore Opt is the size of an optimal solution) when 
taking common patterns belonging to pattern-avoiding classes of permutations. 

1. Introduction and basic definitions 

A permutation tt is said to be a pattern within a permutation a ii a has a 
subsequence that is order-isomorphic to tt. Within the last few years, the study 
of the pattern containment relation on permutations has become a very active 
area of research in both combinatorics and computer science. In combinatorics, 
much research focused on closed classes of permutations, i.e., permutations that 
are closed downwards under forming subpermutations. A huge literature is devoted 
to this subject. To cite only a few of a plethora of suitable examples, Knuth 
considered permutations that do not contain the pattern 312 [15], Lovasz considered 
permutations that do not contain the pattern 213 [17] and Rotem those that do 
not contain 231 nor 312 [19]. 

Surprisingly enough, there is considerably less research on algorithmic aspects 
of pattern involvement. Actually, it appears to be a difficult problem to decide 
whether a permutation occurs as a pattern in another permutation. Indeed, the 
problem in this general version is A'^P-complcte [5]. The case of separable patterns, 
i.e., permutations that contain neither the subpattern 3142 nor 2413, was proved 
to be solvable in 0{kn^) time and 0{kn'^) space in [5], where k is the length of 
the pattern and n is the length of the target permutation. The design of efficient 
algorithms for the recognition of a fixed pattern in a permutation is considered 
in [2] and in particular a 0{n^ logn) time algorithm is given for finding separable 
patterns. L. Ibarra subsequently improved the complexity for separable patterns 
to 0{kn'^) time and 0{kn^) space in [14]. Beside separable patterns, only a few 
restricted cases were considered. A 0(n log log n) time algorithm is presented in [10] 
for finding the longest increasing or decreasing subpermutation of a permutation of 
length n. 

In the present paper we continue this line of research on separable patterns by 
considering the problem of finding a maximum length common separable pattern 
to a set of permutations, i.e., given a set of permutations, find a longest separable 
permutation that occurs as a pattern in each input permutation. Of particular 
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importance in this context, we do not impose here the input permutations to be 
separable. 

This paper is organized as follows. In the remainder of Section 1, we briefly 
discuss basic notations and definitions that wc will use throughout. In Section 2, 
we give a polynomial-time algorithm for finding the largest common separable pat- 
tern that appears as a pattern in a fixed number of permutations. Section 3 is 
devoted to proving hardness of the problem. Finally, some inapproximation issues 
are presented in Section 4. 

1.1. Permutations. A permutation a G Sn is a bijcctive map from [l..n] to itself. 
The integer n is called the length of a. We denote by Oi the image of i under a. A 
permutation can be seen as a word a\a2 . ■ - On containing exactly once each letter 
i G [l..n]. For each entry Oi of a permutation cr, we call i its index and ai its value. 

Definition 1 (Pattern in a permutation). A permutation tt €z Sk is a pattern of 
a permutation a G Sn if there is a subsequence of a which is order-is omorphic to 
n; in other words, if there is a subsequence at-^ai^ . . . dij. of a (with 1 < ii < ^2 < 
. . . < ife < such that < whenever ni < tt^. 

We also say that n is involved in a and call (Ji^^cri^ ■ ■ ■ c-ij, an occurrence of n in a. 

A permutation a that does not contain tt as a pattern is said to avoid tt. 
Classes of permutations of interest are the pattern- avoiding classes of permuta- 
tions: the class of all permutations avoiding the patterns tti , 712 . . . tt^ is denoted 
•^(Tri, 7r2, . . . , TTfe), and S'„(7ri, tt2, ■ ■ ■ ,T^k) denotes the set of permutations of length 
n avoiding tti , 7r2 , . . . , TTfe . 

Example 1. For example a ~ 142563 contains the pattern 1342, and 1563, 1463, 
2563 and 1453 are the occurrences of this pattern in a. But a G S'(321).' a avoids 
the pattern 321 as no subsequence of length 3 of a is isomorphic to 321, i.e., is 
decreasing. 

If a pattern tt has an occurrence (7^^ cr^j . . . ai^, in a permutation cr of length 
n, let / and V be two subintervals of [l..n\ such that {ii,i2, ■ ■ ■ i^k} Q I a-nd 
{ai-^,ai^, . . . ,(Ji^} C V; then we say that tt has an occurrence in a in the intervals 
/ of indices and V of values, or that tt is a pattern of a using the intervals / of 
indices and V of values in a. 

Among the pattern-avoiding classes of permutations, we are particularly inter- 
ested here in the separable permutations. 

Definition 2 (Separable permutation). The class of separable permutations, de- 
noted Sep, IS Sep = S'(2413, 3142). 

There are numerous characterizations of separable permutations, for example 
in terms of permutation graphs [5], of interval decomposition [21, 4, 6], or with 
ad-hoc structures like the separating trees [5, 14]. Separable permutations have 
been widely studied in the last decade, both from a combinatorial [20, 11] and an 
algorithmic [3, 5, 14] point of view. 

We define two operations of concatenation on permutation patterns: 

Definition 3 (Pattern concatenation). Consider two patterns ir and tt' of respec- 
tive lengths k and k' . The positive and negative concatenations of tt and tt' are 
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defined respectively by: 

TT e Tt' = TTl • • • TTfc (ttJ + /c) • • • (tT^, + fc) 

TT e tt' = (tti + k') • • • (tta; + k')T:[ ■ ■ ■ n'^, 

The following property, whose proof is straightforward with separating trees, is 
worth noticing for our purpose: 

Property 1. // both tt and n' are separable patterns, then ttQtt' and ttGtt' are also 
separable. Conversely, any separable pattern tt of length at least 2 can be decomposed 
into TT — tti(3tt2 or tt = ttiQtt2 for some smaller but non-empty separable patterns 
TTi and TT2. 

1.2. Pattern problems on permutations. The first investigated problem on 
patterns in permutations is the Pattern Involvement Problem: 
Problem 1 (Pattern Involvement Problem). 
Input; A pattern tt and a permutation a. 

Output; A boolean indicating whether it is involved in a or not. 

It was shown to be iVP-complete in [5]. However, in [5] the authors also exhibit 
a particular case in which it is polynomial-time solvable: namely when the pattern 
TT in input is a separable pattern. 

Another problem of interest is the Longest Common Pattern Problem (LCP for 
short): 

Problem 2 (LCP Problem). 
Input; A set X of permutations. 

Output; A pattern of maximal length occurring in each a Cz X . 

This problem is clearly A^P-hard in view of the complexity of Problem 1. We 
showed in [6] that it is solvable in polynomial time when X = {(Ti,tT2} with di a 
separable permutation (or more generally, when the length of the longest simple 
permutation [8, 7, 9] involved in ai is bounded). 

In this paper, we will consider a restriction of Problem 2. For any (pattern- 
avoiding) class C of permutations, we define the Longest Common C-Pattern Prob- 
lem (LCCP for short): 
Problem 3 (LCCP Problem). 
Input; A set X of permutations. 

Output; A pattern of C of maximal length occurring in each a G X. 

In particular, we focus in this paper on the Longest Common Separable Pattern 
Problem (LCSepP) which in fact is LCCP where C = Sep. 

To our knowledge, complexity issues of the LCCP Problem are still unexplored. 
We will show in this paper that the LCSepP Problem is TVP-hard in general, but 
solvable in polynomial-time when the cardinality of the set X of permutations in 
input is bounded by any constant K. However the method we use in our polynomial- 
time algorithm for solving LCiSepP on K permutations is specific to separable 
patterns and cannot be extended to any class C of pattern-avoiding permutations. 

Some classes C of permutations are known for which even the Recognition Prob- 
lem (i.e., deciding if a permutation belongs to C) is A^P-hard, so that LCCP on K 
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permutations must be A^P-liard for those classes. [1] gives the example of the class 
of 4-stack sortable permutations. 

However, we are not aware of any example of finitely based pattern-avoiding 
classes of permutations (with a finite number of excluded patterns) for which the 
Recognition Problem is A^P-hard. Thus an open question is to know if the LCCP 
problem for K permutations is polynomial-time solvable for any finitely based C, 
or to exhibit such a class C for which this problem is TVP-hard. 

2. Polynomial algorithm for the longest common separable pattern 
between a finite number of permutations 

In [5], the authors show that the problem of deciding whether a permutation 
TT is a pattern of a permutation a is A'^P-complete. A consequence is that the 
problem of finding a longest common pattern between two or more permutations 
in iVP-hard. However, they describe a polynomial-time algorithm for solving the 
Pattern Involvement Problem when the pattern tt is separable. This algorithm 
uses dynamic programming, and processes the permutation according to one of its 
separating trees. 

With the same ideas, we described in [6] a polynomial-time algorithm for finding 
a longest common pattern between two permutations, provided that one of them is 
separable. Notice that a longest common pattern between two permutations, one 
of them being separable, is always separable. 

In this section, we generalize the result obtained in [6] giving a polynomial-time 
algorithm for finding a longest common separable pattern between K permutations, 
K being any fixed integer, K > 1. Notice that we make no hypothesis on the K 
input permutations. 

Like in [5] and [6], our algorithm will use dynamic programming. However, since 
we do not have a separability hypothesis on any of the permutations, we cannot 
design an algorithm based on a separating tree associated to one of the permutations 
in input. To compute a longest common separable pattern between the input 
permutations, we will only consider sub-problems corresponding to isT-tuples of 
intervals of indices and values, one such pair of intervals for each permutation. 

Namely, let us consider K permutations a^, . . . , , of length tii, . . . , uk respec- 
tively, and denote by n the maximum of the n^'s, 1 < g < K. For computing a 
longest common separable pattern between cr^, . . . , cr^ , we will consider a dynamic 
programming array M of dimension AK , and when our procedure for filling in M 
ends, we intend that M(ii, ji, oi, 6i, . . . , ikiJk, clk, bx) contains a common sepa- 
rable pattern tt between cr^, . . . , that is of maximal length among those using, 
for any q G [1../^], intervals [iq--jq] of indices and [ag..6q] of values in cr''. If we are 
able to fill in M in polynomial time, with the above property being satisfied, the 
entry M(l, ni, 1, rii, . . . , 1, nx, 1, uk) will contain, at the end of the procedure, a 
longest common separable pattern between cr^, . . . , . 

Algorithm 1 shows how the array M can indeed be filled in in polynomial time. 
In Algorithm 1, Longest is the naive linear-time procedure that runs through a set 
S of patterns and returns a pattern in S of maximal length. 

Before giving the details of the proof of our algorithm for finding a longest 
common separable pattern, we state and prove two lemmas. They should help 
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Algorithm 1 Longest common separable pattern between K permutations 
1: Input: K permutations cr^, . . . , of length ni, . . . , uk respectively 

2: Create an array M : 

3: for any integers iq, jq, Qq and bq G [l..nq], for all q E [^--K] do 
4: M(ii, ji,ai,6i, . . . ,iKjK,aK,bK) ^ e 

5: end for 

6: Fill in M: 

7: for any integers Zg, jg, Uq and bq G [l..nq], iq < jq, Uq < bq, for all q e 

by increasing values of J2qUq ^ *g) + i^q ~ ^g) do 
8: if 3q E [l-.A ] such that iq = jq or Uq = bq then 
9: if V(7 e [l..K],3hq G [iq--jq] such that ct/^^ S [aq..6g] then 

10: M{ii,ji,ai,bi, . . ■ ,iK,jK,aK,bK) ^ 1 

11: else 

12: M{ii,ji,ai,bi, . . . ,iKjK,aK,bK) ^ e 

13: end if 
14: else 

15: /*yq G [l..K],iq < jq and a, < bq /* 

M(zi,ji,ai,6i, . . ■ ,iKjK,aK,bK) ^ Longest{S(s U S'e U S") where 

5© = {M[ii,hi-l,ai,ci-l,...,iK,hK -l,aK,CK -l)®M{hi,ji,ci,bi, 

■ ■ ■ ,hK,jK,CK,bK) --iq < hq < jq,aq < Cq < bq,\fq G [l.-K]} 

Sq = {M{ii,hi - l,ci,bi, . . . ,iK,hK - l,CK,bK) G M{hiji,ai,ci - 1, 

■ ■ ■ ,hK,jK,aK,CK - I) ■■ iq < hq < jq,aq <Cq < bq,\fq G [1..A']} 

S = {1} if G [1.. A'], a/i, G such that cr-^^ G [a,.. 

= {e} otherwise. 

16: end if 
17: end for 

18: Output: M{l,ni,l,ni, . . . ,l,nK,l,nK) 



understanding how common separable patterns can be merged, or on the contrary 
split up, to exhibit other common separable patterns. We are also interested in the 
stability of the maximal length property when splitting up patterns. 

Lemma 1. Let tti be a common separable pattern between cr^, . . . , that uses the 
intervals [iq..hq — 1] of indices and [aq..Cq — 1] (resp. [cq..bq\) of values in cr', for 
all q G 

Let TT2 be a common separable pattern between , . . . , that uses the intervals 
[hq..jq] of indices and [cq..bq] (resp. [aq..Cq — 1]) of values in cr', for all q G [l..Ar]. 

Then tt = tti © 7r2 (resp. n = ni Q 1:2) is a common separable pattern between 
cr^, . . . , that uses the intervals [iq-.jq] of indices and [aq..bq] of values in cr', for 
all q G 

Proof. We give a proof for tt = tti © 712 (the case tt = tti © 7r2 being similar). 
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values £ [ai..ci — 1] values S [ci..f)i] 




values £ [a2..C2 — 1] values £ [c2..b2] 



Figure 1. Proof of lemma 1 for = 2 

Fix some q G [L.^J. By hypothesis, there exist occurrences of tti and ■K2 in ct'^ 
the occurrence of tti using the intervals [iq..hq — 1] of indices and [aq..Cq — 1] of 
values, and the occurrence of 7:2 using the intervals [hq..jq] of indices and [cg..6g] 
of values. It is then easily noticed (see Figure 1) that all the elements used in 
one of these occurrences form an occurrence of the pattern tt = tti 7r2 in a'^ in 
the intervals [j<j-.j<j] of indices and [aq..bq] of values. This argument holds for any 
q G [l--^] and hence tt is a common separable pattern between a^, . . . , using 
the intervals [ig-.jq] of indices and [aq..bq] of values in cr'^, for all q e [l.._ftr]. 



□ □ 

Lemma 2. Let tt be a common separable pattern of maximal length between , . . . ,(j 
among those using the intervals [iq..jq] of indices and [aq..bq] of values in a'^ , for 
all q e [1..^]. 

//tt = TTl ©712 (resp. TT — ttiQtt2), with TTl cind TT2 non-empty separable patterns, 
then there exist indices <^n-d values {cq)q)^[i,,K], with iq < hq < jq,aq < 

Cq < bq,yq [1..K], such that: 

i) TTl is a common separable pattern of maximal length between , . . . , among 
those using the intervals [iq..hq — 1] of indices and [aq..Cq — 1] (resp. [cq..bq]) 
of values in a'^ , for all q G [L./ST], and 
ii) TT2 is a common separable pattern of maximal length between , . . . , among 
those using the intervals [hq..jq] of indices and [cq..bq] (resp. [aq..Cq — 1]) of 
values in cr', for all q G [1..A']. 

Proof. Again, consider the case tt = tti (B tt2 (the case tt = tti Q tt2 being similar) . 

Fix some q G By hypothesis, tt = tti (B tt2 has an occurrence in tr' in the 

intervals [iq-.jq] of indices and [ag..5q] of values. By definition of positive pattern 
concatenation, this occurrence splits into two occurrences of tti and tt2 respectively 
(see again Figure 1). More precisely, there exist hq G [iq + l..jq] and Cq G [aq + l..bq] 
such that TTl (resp. TT2) has an occurrence in the intervals [iq..hq — 1] (resp. [hq..jq]) 
of indices and [aq..Cq — 1] (resp. [cg..?),]) of values. This argument holding for all 
q G it becomes clear that tti (resp. TT2) is a common separable pattern 

between a^, . . . , that uses the intervals [iq..hq — 1] (resp. [hq..jq]) of indices and 



LONGEST COMMON SEPARABLE PATTERN BETWEEN PERMUTATIONS 



7 



[aq..Cq — 1] (resp. [cq..6g]) of values in cr', for all q e What remains to prove 

is that TTi and 7r2 are of maximal length among all such patterns. 

Assume that tti is not of maximal length among the common separable patterns 
between cr^, . . . , using interval [iq..hg — 1] of indices and interval [aq..Cq — 1] of 
values in a'' , for all q G [L.-ftT]. Then, there exists 7r^°"^, a common separable pattern 
between cr^, . . . using interval [iq..hq — 1] of indices and interval [aq..Cq — 1] 
of values in cr', for all q S [l-.if], such that |7r^°"^| > |7ri|. Now by Lemma 1, 
^long is a commou separable pattern between cr^, . . . , using the intervals 
of indices and [a^..^^] of values in cr'', for all q G [1..A']. And we have 
I^Zong ^ ^ _ 1^1^ contradicting the maximality of tt. So tti is a 

common separable pattern of maximal length between among those 

using interval [iq..hq — 1] of indices and interval [aq..Cq — 1] of values in a'', for all 
q G In the same way, we prove that is a common separable pattern of 

maximal length between cr^, . . . , among those using interval [hq..jq] of indices 
and interval [cq..^^] of values in ct'', for all q G [l-.A'], ending the proof of the 
lemma. □ □ 

Proposition 1. Algorithm 1 is correct: it outputs a longest common separable 
pattern between the K permutations in input. 

Proof. Consider the array M returned by Algorithm 1. We show by induction on 
Y.qijq-^q) + i^q - "-q) that M(zi , ji , Oi , 61 , . . . , i K , j K , , bx) coutalus a common 
separable pattern tt between a^,...,a^ that is of maximal length among those 
using, for any q G [l..-fC], intervals of indices and [aq..6q] of values in cr'^. 

First, there is no loss of generality in assuming that iq < jq and Qq < bq 
for all q G [l-.i^], since otherwise the above statement is clearly true (indeed, 
M(ii, ji, oi, 61, . . . , i/f , jif , a/f , ) contains e and either = or [aq..6g] = 

for some q G [L.ii']). 

If J2qUq ^ + (bq ~ a?) ~ 0, then iq = jq and Uq ~ bq for all q G [1..A']. Con- 
sequently, the pattern we would like M(«i, ji, ai, 6i, . . . , iK,3K, o-k, bx) to contain 
is a longest common separable pattern between tr^, . . . ,cr^ that uses only index 
iq ~ jq s-nd value Oq ~ bq in cr' for all q G [l..isr]. Such a pattern is either e or 
1. And it is 1 if and only if Vg G [l..K],a1 ~ Oq that is to say if and only if 
Vg G [l..K],3hq G [iq--jq] such that a^^ G [aq..bq]. On hues 8 to 12 of Algorithm 1, 
we see that in this case M(ii,ji, ai, 6i, . . . , ixijK, cik, bx) is set correctly. 
'^qi.'jq ^ iq) + {bq — Oq) > 0, we must consider two subcases: 

If 3g G [l..if] such that iq = jq or Oq = bq, let us call tt a common sep- 
arable pattern between cr^,...,(T^ that is of maximal length among those us- 
ing, for any q G [l.-AT], intervals [iq..jq] of indices and [aq..bq] of values in cr'. 
Then, just as before, tt is either 1 or e, because in at least one of the permuta- 
tions, say cr'', the occurrence of tt can use at most one index (if iq = jq) or at 
most one value (if aq ~ bq). And again, lines 8 to 12 of Algorithm 1 show that 
M(ii, ji, ai, 6i, . . . , Zif, jK, a/f , 6x) contains 1 or e. More precisely, the condition 
on line 9 ensures that it contains 1 exactly when 1 has an occurrence in cr' in the 
intervals [jg-.jq] of indices and [aq..bq] of values, for any q G [1..A']. 

It remains to consider the recursive case when Vg G [l..i^],iq < jq and Oq < bq. 
In this case, consider tt a common separable pattern tt between ct^, . . . , that is 
of maximal length among those such that, for any q G [l.-if], tt has an occurrence 
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in cr' in the intervals [jg.-jg] of indices and [ag..6g] of values. Since tt is separable, 
then either tt = e, or tt = 1, or tt = tti © 7r2, or tt = tti 7r2 where tti and 7r2 are 
smaller but non-empty separable patterns. Now consider ■Kaigo a- longest pattern in 
the set SqU SqU S where 

Sfs = {M{ii,hi-l,ai,Ci-l,...,iK,hK~l,aK,CK-l)(S)M{hi,ji,Ci,bi, 

■ ■ ■ ,hK,jK,CK,bK) ■■iq<hq< jq,aq < Cq < bq,\fq G [l.-K]} 
Sq = {M{ii,hi - l,ci,bi, . . . ,iK,hK - l,CK,bK) G M{hiji,ai,ci ~ 1, 

. . . , hK,jK, aK,CK ~ I) ■■ iq < hq < jq, aq < Cq < bq^q e 
S = {1} if e [l..K],3hq e [iq-iq] such that a^^ e [aq..bq], 
= {e} otherwise. 

By induction hypothesis, each entry of M that appears in the set U Sq is 
a common separable pattern between ct^, . . . , whose occurrence in cr', for any 
q e [l.._ftr], uses indices and values in the prescribed intervals, and that is of maximal 
length among all such patterns. Notice also that = 1 if and only if 1 has an 
occurrence in a'^ in the intervals of indices and [ag..6g] of values, for all q G 

An immediate consequence of those two facts and of Lemma 1 is that TTaigo 
is a common separable pattern between cr^, . . . , which has, for any q G [l..i4r], 
an occurrence in a'^ in the intervals of indices and [ag..5q] of values. What is 

left is to prove that |7rtjigo| = |7r|. This is clear when tt = e or 1. So assume that tt = 
7ri©7r2, the case tt = 7riG7r2 being very similar. Since tt = 7ri©7r2 has an occurrence 
in each a'^ in the intervals [ig.-jg] of indices and [ag..6g] of values, by Lemma 2, 
there exists indices (^q)ge[i.._fs-] and values (cq)gg[i. jfj, with iq < hq < jq,aq < 
Cq < bq,yq G [L.X], such that tti has an occurrence in each cr'' in the intervals 
[iq..hq — 1] of indices and [aq..Cq — 1] of values and tt2 has an occurrence in each a'' 
in the intervals [hq..jq] of indices and [cg..6q] of values. Lemma 2 also states that tti 
and TT2 are of maximal length among the common separable patterns in the given 
intervals of indices and values. So by induction hypothesis, \M(ii,hi — l,ai,ci — 
l,...,iK,hK - l,aK,CK - 1)1 = kil and \M[hi,ji,ci,bi,...,hK,iK,CK,bK)\ = 
|7r2|. Consequently, \tt\ = |7ri©7r2| = /ii — 1, ai, ci — 1, . . . , ik, hK~l,aK, ck — 

l)\ + \M{hi,ji,ci,bi, - ■ ■ ,hK,jK,CK,bK)\ < \i^aigo\- The inequality |7r| > |7ra;go| 
being obvious by maximality of tt, we conclude that |7r| — \i:aigo\- This ends the 
proof in the case tt = tti © 7r2. For the case tt = tti © 7r2, the proof follows the exact 
same steps, with tti having an occurrence in each a'^ in the intervals [iq..hq — 1] of 
indices and [cg..6q] of values and tt2 having an occurrence in each cr' in the intervals 
[hg..jq] of indices and [aq..Cg — 1] of values. □ □ 

Proposition 2. Algorithm 1 runs in time 0(n^^+^) and space 0{n'^^^'^). 

Proof. Algorithm 1 handles an array M of size 0{n'^^), where each cell contains 
a pattern of length at most n, so that the total space complexity is 0{n'^^'^^). 
For filling in one entry M{ii, ji,ai,bi, . . . ,iK, Jk, o.k, ), if 3(7 G [L.-R'] such that 
iq = iq or «9 = bq (hues 9 to 13 of Algorithm 1), the time complexity is 0{n^). 
If no such q exists (line 15 of Algorithm 1), the time complexity needed to fill in 
Af(ii, ji, fli, 6i, . . . ,iK,jK,aK,bK), using the entries of M previously computed, is 
0{n^^~^^). Indeed, we search for an element of maximal length among 0{n^^) 
elements, each element being computed in C(n)-time as the concatenation of two 
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Figure 2 . Shown here is the correspondence between the permu- 
tation TT = 2 3 5 4 1 and the associated matching diagram G. 

previously computed entries of M. Consequently, the total time complexity to fill 
in M is □ □ 

A consequence of propositions 1 and 2 is: 

Theorem 1. For any fixed integer K , the problem of computing a longest common 
separable pattern between K permutations is in P. 

We may wonder whether a longest common separable pattern between two per- 
mutations (T^ and CT^ (computed in polynomial time by Algorithm 1) is a good 
approximation of a longest common pattern between and (whose computa- 
tion is iVP-hard). Section 4 gives a negative answer to this question, by the more 
general Corollary 1. 

3. Hardness result 

We proved in the preceding section that the LCiSepP problem is polynomial-time 
solvable provided a constant number of input permutations. We show here that the 
CSepP problem (the general decision version of LC5epP), is A^P-complete. 
Problem 4 {CSepP Problem). 
Input; A set X of permutations and an integer k. 

Output; A boolean indicating if there a separable pattern of length k occurring in 
each a £ X . 

Actually, we will prove more, namely that the CSepP problem is iVP-complete 
even if each input permutation is separable. An immediate consequence is the NP- 
hardness of hCSepP. For ease of exposition, our proof is given in terms of matching 
diagrams. 

Definition 4 (Matching Diagram). A matching diagram G of size n is a vertex- 
labeled graph of order i.e., number of vertices, 2n and size i.e., number of edges, n 
where each vertex is labeled by a distinct label from {1,2,..., 2n} and each vertex 
i G {1, 2, . . . , 7i} (resp. j G {?! + 1, 7i + 2, . . . , 2n} ) is connected by an edge to exactly 
one vertex j d {n + l,n + 2, . . . , 2n} (resp. i G {1, 2, . . . ,n}). We denote the set of 
vertices and edges of G by V{G) and E{G), respectively. 

It is well-known that matching diagrams of size n are in one-to-one correspon- 
dence with permutations of length n (see Figure 2 for an illustration). 

Let G and G' be two matching diagrams. The matching diagram G" is said to 
occur in G if one can obtain G' froqi G by a sequence of edge deletions. More 
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formally, the deletion of the edge {i,j), i < j, consists in (1) the deletion of the 
edge (2) the deletion of the two vertices i and j, and (3) the relabeling of all 

vertices k G [i + — 1] to /c — 1 and all the vertices k > j to k — 2. Therefore, the 
decision version of the LC5epP is equivalent to the following problem: Given a set 
of matching diagrams and a positive integer fc, find a matching diagram of size k 
which occurs in each input diagram [16]. 

Clearly, two edges in a matching diagram G are either crossing o''?"^ or 

nested "^^^^^ , Moreover, it is easily seen that an occurrence in G of a matching 
diagram G" of which all edges are crossing (resp. nested) correspond to an occur- 
rence in the permutation associated with G of an increasing (resp. decreasing) 
subsequence. 

For the purpose of permutations, convenient matching diagrams are needed. 
A matching diagram is called a tower if it is composed of pairwise nested edges 

''^^^ and a staircase if it is composed of pairwise crossing edges ^■^^'^ . A 
matching diagram is called a tower of staircases if its edge set can be partitioned 



in nested staircases 

Theorem 2. The CSepP problem is NP-complete even if each input permutation 
is separable. 

Proof. GSepV is clearly in NP. For proving hardness, we reduce from the Independent- 
Set problem which is known to be iVP-complete [13] . Let G be an arbitrary graph 
instance of the Independent-Set problem. Write V[G) = {1,2, .. . ,n}. We now 
detail the construction of n + 1 matching diagrams Gq, Gi, G2, . . . , G„, each cor- 
responding to a separable permutation. First the matching diagram Go is a tower 
of n staircases Aq, 1,^0,2, ■ • ■ ,^o,n, each of size n -\- 1 (see Figure 3, middle part; 
staircases are represented by shaded forms), i.e., 



Each matching diagram Gi, 1 < i < is composed of two crossing towers of n 
staircases each referred as Ai^i, Ai^2, ■ ■ • , ^i,n and Bi^i, -6^,2, • ■ • , Sj,n (see Figure 3, 
bottom part), and defined as follows: 



It is simple matter to check that all matching diagrams G;, < i < n, correspond to 
separable permutations and that our construction can be carried on in polynomial 
time. This ends our construction. 

We claim that there exists an independent set V C V{G) of size k in G if and 
only if there exists a matching diagram Gsoi of size ri^ + k that occurs in each input 
matching diagram Gi, < i < n. 

Suppose that there exists an independent set V' C V{G) of size k in G. Consider 
as a solution Gsoi the tower of n staircases Gi, G2, . . . , G„ (see Figure 3, top part) 
of total size + k, where the size of each staircase is defined according to the 
following: |Gi| — n if i ^ V' and \Ci\ ~ n + 1 if i G V' . We claim that Ggoi occurs 



Vj, I <j <n, 



\Aoj \ = n + 1. 
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in each input matching diagram Gi, < i < n. Indeed, by construction, for any 
i G V{G), ii i then Gsoi occurs in Side-A of d and if i € then Gsoi occurs 
in Side-_B of Gi. To complete the claim, we note that Gsoi occurs in Go (Gq is 
indeed a tower of n staircases, each of size n + 1). 

Conversely, suppose that there exists a matching diagram Gsoi of size + k that 
occurs in each input matching diagram Gi, < i < n. Let us prove the following: 

Claim 1. Gsoi is a tower of n staircases. Furthermore, for any i G [l.-?^], Gsoi 
occurs completely in Side- A or completely in Side-B in Gi. 

Proof. Let us first focus on an occurrence of Gsoi in Gq. Since Go is a tower of 
staircases, then it follows that Gsoi is a tower of staircases as well. We now prove 
that Gsoi is a tower of n staircases. Indeed, suppose, aiming at a contradiction, 
that Gsoi is a tower of n' < n staircases. Then it follows that Gsoi has size at most 
(n + l)(n — 1) = — 1 < + k. This is the desired contradiction, and hence 
Gsoi is a tower of n staircases Gi, C2, . . . , G„, each of size at most n + 1. We now 
turn to considering an occurrence of Gsoi in Gi, 1 < i < n. We prove that Gsoi 
occurs completely in Side-A or completely in Side-_B (see Figure 3). Suppose, for 
the sake of contradiction, that Gsoi matches at least one edge in Side~A, say e, and 
at least one edge in Side-i3, say e'. By construction, e and e' are crossing edges in 
Gi , and hence e and e' are matched by two edges that belong to the same staircase 
in Gsoi (Gsoi is indeed a tower of staircases). We now observe that any edge in 
Side-j4 crosses the edge e' and any edge in Side-B crosses the edge e. Then it 
follows that the occurrence of Gsoi in G; induces a single staircase. But Gsoi is a 
tower of n staircases, each of size at most n + 1. A contradiction. Therefore, in any 
matching diagram Gi, I < i < n, Gsoi occurs completely in Side-A or completely 
in Side-B. □ 

As an important consequence of the claim, there is thus no loss of generality in 
assuming that each staircase of Gsoi has size n or n+ 1, and hence Gsoi is composed 
of n — fc staircases of size n and k staircases of size n + 1 (since Gsoi has size + k). 
Consider now the subset V' C V{G) defined as follows: i G V{G) belongs to V if 
and only if |Gi| = n + 1, i.e., the staircase Ci has size n + 1 in Gsoi- According to 
the above, V' has certainly size k. We shall show that V' is an independent set in 
G. Indeed, let i £ V and consider the matching diagram Gi. Since i E V then it 
follows that \Ci\ =71 + 1, and hence Gsoi occurs in Side-_B in Gi (the latter follows 
from the fact that \Ai,i\ ~ n). But, by construction, for \ < j < n, \Bi,j \ = n + 1 
if and only if (i, j) is not an edge in G (in particular, = n + 1). Hence, the 

vertex i is not connected to any vertex in V . Therefore, since the argument applies 
to each matching diagram Gi with i £ V' , V' is an independent set in G. □ □ 



In this section, we return to the LCCP Problem for K permutations. As said 
before, the general LCP Problem is NP-hard as well as the Pattern Involvement 
Problem [5]. In this section we prove the following result: 

Theorem 3. For all e > and C, a pattern-avoiding class of permutations, there 
exists a sequence ((T„)„gN of permutations On G Sn such that 



4. Approximation ratio 
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n staircases 
each of size ti or re + 1 




Side-B 



Figure 3. Reduction in the proof of Theorem 2 

where 7r„ is the longest pattern of class C involved in cr„ . 

Before proving this result we need the following Lemma. 

Lemma 3. Given a ■permutation tt £ Sk, the number of permutations a £ Sn such 
that IT is involved in a is at most (n — • 

Proof. Let tt = 'Ki'K2 • • ■ , ""fc and a ~ (Ti(T2 . . . cr„ be such that tt is involved in a. 
There exist ii < i2 < ■ ■ ■ < ik such that (Ji^^ai^ ■ ■ -Ci^ is order-isomorphic to tt. 
Then a = ui(Ji-^U2<Ji2 ■ ■ ■ Uk<Ji^Uk+i where Ui is a factor of a. We will call iti's the 
blocks associated to cr, tt and an occurrence of tt in a. 

Conversely, suppose we are given a permutation n £ Sk, and ui, U2, . . . , Ufe+i k+1 
sequences of distinct numbers in {1 . . .n} such that |mi| + |u2| + . . . + ~ n — k. 

Let E = {ni, n2, ■ ■ - rik} the k integers in {1 . . . n} which do not appear in a block 
Ui. We denote by j the j*'' smallest element in E. Then the permutation a = 
uiTf[u2TT2 ■ ■ ■ TFfcUfe+i is a permutation of Sn and tt is involved in a. For example, if 
TT = 2143, n = 9, Ml = 31, M2 = 0, M3 = 8, U4 = 65 and M5 = then E = {2, 4, 7, 9} 
and a = 314289657. 

Note that two different lists of blocks could lead to the same permutation cr if tt 
has several occurrences in a as shown in Figure 4. 

Thus, the number of permutations a such that tt is involved in a is at most 
the number of different lists of blocks. There are („"j,) different choices for the 
numbers that appear in one of the ut. Then t^ere are (n — fc)! different orders for 
these numbers. The last step is to cut the word so obtained into k + 1 (possibly 
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{(1,0,0,56,0) 
(1,0,4,6,0) — > 1374562 
(1,0,45,0,0) 

Figure 4. Relation between pattern, blocks and permutation 

empty) blocks ui,M2, ■ • -Wfc+i- There are such choices. Hence we have the 

claimed formula. □ □ 

We can now prove Theorem 3. 

Proof. We make the proof by contradiction. We first prove that if the result were 
false, every permutation of length n would contain a pattern of C of length ex- 
actly k — \nP'^^'^~\. Next, we show that the number of permutations of length n 
containing one permutation of C p| Sk as a pattern is strictly less than n\. 

Suppose that there exist e > and C a pattern-avoiding class of permutations 
such that for every permutation a Cz Sn, the longest pattern tt G C of cr has length 

> ri'''l°'^^^l = ^- As C is closed - every pattern tt of a permutation cr G C is also 
in C- for every permutation a € Sn there exists a pattern tt G C of cr whose length 
is exactly \tt\ = k. 

But the number of permutations in Cf^Sk is at most c*^ by [18]. By Lemma 
3, for each permutation in Cf^Sk, the number of permutations in Sn having this 
permutation as a pattern is at most (n — A:)!(^) . Thus the number of permutations 
in Sn having a pattern in C p| 5>fe is at most c'^(n — fc)! (^)^. But with k = , 
0*^(71 — /c)!(^')^ = o ^7i"^ = o(?i!). Note that a similar proof is given in [12] for 
finding the smallest permutation containing all patterns of a given length. □ □ 

Corollary 1. The LCP Problem cannot be approximated with a tighter ratio than 
\JOpt by the LCCP Problem, where C is a pattern- avoiding class of permutations, 
and Opt is the size of an optimal solution to the LCP Problem. 

Proof. Consider the LCP Problem between a and a. Then the optimal solution to 
the LCP Problem is cr. But the solution to the LCCP Problem is a longest pattern 
of cr belonging to the class C. By Theorem 3, such a pattern may have size \/\o] 
asymptotically. □ □ 
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